Finite temperature Mott transition in Hubbard model on anisotropic triangular lattice 
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We investigate the Hubbard model on the anisotropic triangular lattice by means of the cellular 
dynamical mean field theory. The phase diagram determined in the Hubbard interaction versus tem- 
perature plane shows novel reentrant behavior in the Mott transition due to the competition between 
Fermi-liquid formation and magnetic correlations under geometrical frustration. We demonstrate 
that the reentrant behavior is characteristic of the Mott transition with intermediate geometrical 
frustration and indeed consistent with recent experimental results of organic materials. 

PACS numbers: 71.30.+h 71.10.Fd 71.27.+a 
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Geometrical frustration has attracted much interest in 
the field of strongly correlated electron systems. The 
discovery of heavy fermion behavior in the pyrochlorc 
compound L1V2O4 [lj and the superconductivity in the 
triangular lattice compound Na x Co02 • yH 2 2\ have 
stimulated intensive studies of frustrated electron sys- 
tems. Geometrical frustration has also uncovered new as- 
pects of the Mott metal- insulator transition, which is now 
one of the central issues in the physics of strongly corre- 
lated electron systems. In particular, recent experiments 
on the triangular lattice organic materials k-(BEDT- 
TTF)2X have revealed various interesting physics, such 
as a novel spin liquid state in the Mott insulating phase, 
unconventional superconductivity, etc [E 0, HI- Since 
the path integral renormalizationgroup study of the tri- 
angular lattice Hubbard model [6(, the correlated elec- 
trons on the anisotropic triangular lattice have been in- 
tensively studied so far @, S, H EE EL El EE EI EE Ell • 
Geometrical frustration effects on the finite-temperature 
(T) Mott transition, however, have not been sufficiently 
understood yet. One of interesting and nontrivial fea- 
tures in the finite-T Mott transition is reentrant behavior 
observed in the frustrated organic material k-(BEDT- 
TTF) 2 Cu[N(CN) 2 ]C1 under pressure [E i]. With low- 
ering temperature, it once undergoes a transition from 
Mott insulator to metal, and then reenters the paramag- 
netic insulating phase at a much lower temperature. This 
reentrant behavior is quite different from the Mott tran- 
sition in the three dimensional systems, such as V2O3, 
and is expected to be a new aspect due to the geometri- 
cal frustration and/or low-dimensionality. 

The dynamical mean field theory (DMFT) has given 
substantial theoretical progress in understanding the 
Mott transition, but this method, not treating spatial 
fluctuations, does not explain the reentrant behavior: In 
DMFT, the metallic phase always lies on the lower tem- 
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FIG. 1: Phase diagram of Hubbard model on anisotropic 
triangular lattice for t'/t = 0.8. 



perature side of the finite-T Mott transition line [l8( ■ It is 
thus desirable to properly incorporate spatially extended 
correlations and geometrical frustration by employing a 
different appropriate method. 

In this paper, we study the finite-T Mott transition in 
the Hubbard model on the anisotropic triangular lattice 
by means of a cluster extension of DMFT, the cellular- 
DMFT (CDMFT) (HHl. The obtained phase diagram 
indeed shows a reentrant behavior in the Mott transition 
(see Fig. [TJ, which is consistent with that observed in 
the organic material mentioned above. We argue that 
the reentrant behavior is a generic feature inherent in 
the Mott transition with intermediate geometrical frus- 
tration, and thus can be found in various frustrated elec- 
tron systems experimentally. 

We consider the standard Hubbard model on the trian- 
gular lattice with the hopping anisotropy (see Fig. (Ha)), 
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with riiu = c\ a Ci a , where c icr (cj a ) creates (annihilates) 
an electron with spin a at site i. To analyze this model 
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FIG. 2: (a) Sketch of the anisotropic triangular lattice and (b) 
the effective cluster model using four sites cluster CDMFT. 



we use CDMFT, which has been successfully applied to 
frustrated systems such as the Hubbard model on the tri- 
angular lattice 11, TBI, 17 1 and the Kagome lattice 21 1. 
In CDMFT, the original lattice is regarded as a superlat- 
tice consisting of clusters, which is then mapped onto an 
effective cluster model via a standard DMFT procedure. 
Considering four sublattices labeled by 1-4, as shown in 
Fig. [U^a), the original lattice model is converted into 
a four-site cluster model coupled to the self-consistently 
determined medium illustrated in Fig. HJb). Given the 
Green's function for the effective medium, Q, we can com- 
pute the cluster Green's function G and the cluster self- 
energy E by simulating the effective cluster model with 
Quantum Monte Carlo (QMC) method ;23j. Here, G, 
G, and E are described by 4 x 4 matrices. The effective 
medium Q is then computed via the Dyson equation, 
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where /i is the chemical potential and t (K) is the Fourier- 
transformed hopping matrix for the sublatticc. Here, 
summation of K is taken over the reduced Brillouin zone. 
We repeat this procedure until the results are converged 
(fifty times iterations are enough). In each iteration, we 
typically use 10 6 QMC sweeps and Trotter time slices 
L = 12£/T, where time slice errors are reduced by inter- 
polation scheme based on a high-frequency expansion of 
Green's function [24j |. 

Let us now study the finite-T Mott transition in the 
model (JlJ at half filling. We first investigate the In- 
dependence of the double occupancy T> occ . — {ritfriii) 
for interaction strength U/t = 8, where the ground state 
is insulating and in the vicinity of Mott transition point. 
A remarkable property in our frustrated system is non- 
monotonic T-dcpcndence of -D occ .. As shown in Fig. [3j 
-Docc. decreases at high-T, and then shows an upturn at 
intermediate-T taking a local minimum at T/t ~ 0.5, as 
T decreases. At much lower T, D occ , starts to decrease 
again, showing a hump structure. This nonmonotonic 
T-dependence of -D occ . indicates that the system once 
changes from insulating to metallic and then reenters the 
insulating phase as T is lowered. The behavior is quite 
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FIG. 3: Nonmonotonic temperature dependence of double 
occupancy for U/t = 8.0, where ground states are insulating 
and in the vicinity of the Mott transition. Arrows denote 
the crossover temperature from the high-T insulator to the 
intermediate-T metal. 



different from that in the infinite dimensional Hubbard 
model, in which D occ _(T) has a single minimum at the 
Fermi-liquid (FL) coherence temperature T . In the lat- 
ter case, the system is insulating at T > T and FL at 
T < To [18j . The nonmonotonic behavior in our system is 
also different from that in the unfrustrated square lattice 
Hubbard model. On the square lattice, the FL coherence 
is disturbed by the antiferromagnetic (AF) interaction 
due to the perfect nesting, which gives rise to monotonic 
HI, 2fi| • Note that the hump structures 



decrease of D occ , 
in D occ . become more prominent and shift to lower tem- 
peratures as t' increases, although they get smeared for 
t'/i = 0.5. Therefore, the nonmonotonic behavior is a 
characteristic feature caused by geometrical frustration. 

To elucidate whether the change between metal and in- 
sulator is a real phase transition or crossover, we further 
investigate the double occupancy for typical anisotropy 
t'/i = 0.8 with varying U. We start from the non- 
interacting system to reach the large-C/ regime, typi- 
cally U/t — 10, and then calculate T> occ with decreas- 
ing U gradually. As shown in Fig. [5J the double oc- 
cupancy jumps at critical interaction strength U c with 
decrease of U, which signals the first order Mott transi- 
tion. The jump shrinks with increasing T, and vanishes 
above T/t ~ 0.3. The critical end point is expected to 
be located at T/t - 0.3 and U/t ~ 9.1. At high-T, 
the system shows a crossover between metal and insula- 
tor, where we define the boundary by the temperature 
at which the double occupancy takes the first local mini- 
mum at high-T (see Fig. |3|). This boundary is consistent 
with that determined by the local minimum of the den- 
sity of states at the Fermi energy. The phase diagram 
thus determined is drawn in Fig. [TJ The remarkable 
point is that U C (T) in our system shows a slope with the 
opposite sign to that in the infinite dimensional Hubbard 
model at low temperatures, whereas the high-T crossover 
shows similar behavior to the well-known results in infi- 
nite dimensions. 
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FIG. 4: Double occupancy as a function of interaction 
strength U/t for t' /t = 0.8 at several temperatures. We show 
only the transition from insulator to metal with weakening U 
(defined as U c i), although we find hysteresis. 



Within the single-site DMFT, entropy per site in the 
paramagnetic insulating phase is roughly In 2 correspond- 
ing to localized free spins and the metallic phase with the 
FL coherence has smaller entropy. Hence, in the vicinity 
of Mott transition temperature, the system becomes in- 
sulating at high-T to gain the entropy and the metallic 
phase always lies in the lower temperature regime [lij . 
However, the spatial fluctuations, which are not incorpo- 
rated in the single-site DMFT, become important at low 
temperatures. For example, the dynamical cluster treat- 
ment of the unfrustrated square lattice shows that the FL 
metallic phase does not appear because of strong AF cor- 
relations [25|, [26|. On the other hand, in our system, the 
magnetic correlations are hard to develop down to low 
temperature T/t ~ 0.4 because of geometrical frustra- 
tion. Therefore, as temperature decreases, the entropy is 
released not by spin correlations but by the itinerancy of 
electrons in T/t > 0.4, which causes the crossover from 
insulator to metal in Fig. [TJ The appearance of this kind 
of FL states is one of characteristics near the Mott tran- 

At much lower 



sition with geometrical frustration 21 1. 
temperatures, the magnetic correlations get enhanced, 
which finally trigger the first order transition from the FL 
to the insulator with smaller entropy. Therefore, as T de- 
creases, U C (T) decreases at low temperatures (T/t < 0.4) 
in contrast to at high temperatures (T/t < 0.4). We 
thus conclude that the reentrant Mott transition on the 
anisotropic triangular lattice is due to the competition in- 
duced by geometrical frustration between the FL forma- 
tion and the magnetic correlations. Since such competi- 
tion is expected to happen in frustrated electron systems 
in low dimensions, the reentrant behavior in the Mott 
transition could be found in a wide variety of frustrated 
materials. 

We can clearly see the development of the quasiparti- 
cles and magnetic correlations discussed above in the mo- 
mentum resolved one-particle spectrum A^(uj). Within 
CDMFT, the one-particle Green's function is given as, 




FIG. 5: Momentum resolved one-particle spectrum Ak(w) 
for U/t = 8.0, t'/t — 0.8 at several temperatures. 



where k is the wave vector in the original Brillouin zone 



G k (w) = I^y k ' (r -- ri) 

7,5 



w + /x-t(k) -E(w) ,(3) 



7<5 



and r 7 , rs label four cluster sites 



We calculate the 
imaginary time Green's function Gk (t) and obtain the 
spectrum Ak (lu) = — ImGk (uj + iO) /ir using the maxi- 
mum entropy method ji^. In Fig. EJ we show Ak (lo) for 
U/t = 8, t'/t = 0.8 at typical temperatures. At high tem- 
peratures Ak (uj) shows an insulating behavior, where it 
has a large Hubbard gap of order of U ft and no quasipar- 
ticle peak. As temperature is lowered, the quasiparticle 
peak starts to develop inside the gap, having weak dis- 
persion, which clearly demonstrates the emergence of the 
frustration-induced metallic phase. The stabilization of 
the metallic phase by geometrical frustration is consistent 
with the previous studies of the Hubbard model on the 
triangular lattice [1, H^] and the Kagome lattice 21 1 . At 
much lower temperatures, the quasiparticle peaks split 
and acquire a very small gap, where the system is in- 
sulating again. This small gap is due to the magnetic 
exchange interaction among the quasiparticles, which is 
consistent with the results at zero temperature recently 
obtained by CDMFT with exact diagonahzation method 
[lB | . We thus confirm that our frustrated system exhibits 
the insulator-metal-insulator reentrant behavior as tem- 
perature decreases. 

Finally we examine the magnetic instability by calcu- 
lating the static spin susceptibility \ q . We find that the 
spin susceptibility \q remains finite just below the first 
order Mott transition temperature and hence the param- 
agnetic Mott insulator is not precluded by the magneti- 
cally ordered phase in contrast to the single-site DMFT 
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FIG. 6: Static spin susceptibility \q m t ne insulating phase 
for U/t = 9, t'/t = 0.8 at T/t = 0.2. 



where the magnetic order almost conceals the Mott tran- 
sition [l8|, (28(. Although the appearance of a finite- 
temperature magnetic transition is due to a mean-field 
type approximation, the magnetic ordering in our case, 
highly suppressed by frustration effects, appears below 
the Mott transition temperature, fn Fig. [SI we show \ q 
for U/t = 9, t'/t = 0.8 at T/t = 0.2, where the system is 
in the insulating phase close to the Mott transition point. 
The susceptibility \q takes a maximum at incommensu- 
rate wave vectors q ~ (0.77T, 0.77r) and does not diverge. 
The magnetic correlations thus play an important role in 
driving the Mott transition at low temperatures, but do 
not induce a real magnetic instability due to strong frus- 
tration. We note that our calculations of \ q for different 
values of t' are also consistent with recent results of the 
Hubbard model around the Mott transition 0, 0, [H, 2A I : 
For large-t/ and weak frustration t'/t < 0.7, Xq nas a 
peak at q — (ix, ir) corresponding to the commensurate 
antifcrromagnetism, which rapidly develops with lower- 
ing T. We also find that \ q for t' ft = 1 takes a maximum 
at q = (27r/3, 2tt/2>) corresponding to the 120° structure, 
in which the development of Xq is much slower than the 
AF correlations for small t' . 

In summary, we have investigated the Mott transition 
in the Hubbard model on the anisotropic triangular lat- 
tice by means of CDMFT. The obtained phase diagram 
(Fig. [J) in the U-T plane for t'/t = 0.8 is qualita- 
tively consistent with the reentrant Mott transition in 
k-(BEDT-TTF) 2 Cu[N(CN) 2 ]C1 with t'/t ~ 0.75. By cal- 
culating the wave- vector dependent spectral function and 
susceptibility, we have clarified that the reentrant-type 
Mott transition is due to the competition between the FL 
formation and the magnetic correlations under geometri- 
cal frustration. We have also studied the Mott transition 
for different values of t'/t. For weakly frustrated case 
t'/t < 0.5, we have found that the FL formation is sup- 
pressed by the strong AF correlations, so that the reen- 
trant Mott transition becomes obscured. On the other 
hand, we have found that for fully frustrated case t' ~ t, 
the FL states are well stabilized by frustration and the 
metallic region is extended, because the magnetic fluctu- 
ations of the 120° structure are weak. Hence, the low-T 



Mott transition line shifts to much lower-T regime 
The above tendency is consistent with another organic 
material k-(BEDT-TTF) 2 Cu 2 (CN) 3 with t' ~ t, where 
magnetic ordering was not observed down to the low- 
est temperature studied experimentally. Therefore the 
reentrant behavior can be most clearly observed in mod- 
erately frustrated electron systems. Although we have 
used the four-site-cluster CDMFT, we believe that the 
reentrant behavior of Mott transition is robust and our 
results do not qualitatively change even for a larger clus- 
ter size, because the competition between FL formation 
and magnetic correlations should occur generally in frus- 
trated electron systems. We expect that the reentrant 
behavior in the Mott transition found here will be ob- 
served experimentally in a variety of frustrated electron 
systems. 
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